chore(e2e): run Playwright visual tests in Docker (#DS-5311) - #1889
Conversation
Backport of #1845 and #1886 from main. The screenshot baselines are compared with threshold: 0 and carry no {platform} suffix, so they belong to one OS and one browser build. Until now a developer could neither run nor regenerate them off CI, and CI itself was pinned only by whatever ubuntu-latest meant that week. Deliberate divergences from main, none of which have a counterpart there: - the base image is pinned to the v1.55.0-noble digest, matching this branch's Playwright rather than main's 1.62.1; - @playwright/test drops its caret. It already resolved to 1.55.0, so nothing installed changes, but tools/e2e/run.js needs an exact version to name an image tag, and a patch bump moves the bundled Chromium and invalidates every baseline; - packages/cdk joins the build-context allowlist. 73 files under packages/components import @koobiq/cdk/{a11y,keycodes,testing}; on main the CDK lives inside components/core and needs no entry; - the ignore rules land in .eslintignore, since this branch predates the flat config; - everything about the docs smoke suite is dropped - there is no e2e:docs script and no playwright.docs.config.ts here. No baseline is regenerated: all 182 pass unchanged inside the container.
Both captured the page before it had finished assembling itself, and both were exposed by running the suite in Docker, where a developer machine drives more workers against one dev server than a 4-vCPU runner does. code-block reaches highlight.js through a dynamic import and rewrites each block's innerHTML as it resolves, stamping data-language on the element it has finished. A screenshot taken before that lands catches the page part highlighted, which changes the element's height rather than a few pixels and so reports as "Expected an image 1556px by 3540px, received 1556px by 3232px" - a layout regression, to read it. It is also marked slow(): with fifteen blocks to highlight and two captures of a ~3500px-tall element it is the heaviest test in the suite, and under contention it was overrunning the 15s default and failing on the budget with no screenshot to show why. icon fetches every glyph from /assets/SVGIcons; an unresolved host occupies no space, so the whole page reflows as they arrive and the diff is a horizontal text shift rather than a wrong-looking icon. The open dropdown made it worse by adding an overlay animation on top, and is dropped here as it was on main in #1848 - 02-light.png is regenerated for that. Neither wait is used on E2eIconStateAndStyle: it registers no icon resolver, renders its icons as font classes, and would never satisfy the assertion.
There was a problem hiding this comment.
Pull request overview
This PR makes Playwright visual regression runs reproducible across developer machines and CI by running the suite inside a pinned Playwright Docker image, and updates CI + docs to standardize on that flow.
Changes:
- Add a Docker-based Playwright runner (Dockerfile + compose + wrapper) and wire it into CI and snapshot-approval workflows.
- Pin
@playwright/testto an exact version and validate Docker image/browser alignment to keep screenshot baselines stable. - Reduce visual-test flakiness by waiting for async rendering (icons loading, code highlighting) before taking screenshots.
Reviewed changes
Copilot reviewed 19 out of 21 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| yarn.lock | Updates lockfile entries to reflect exact @playwright/test pin. |
| package.json | Pins @playwright/test and adds e2e:docker scripts. |
| tools/e2e/run.js | Adds the local/CI entrypoint wrapper that runs the suite via docker compose. |
| tools/e2e/Dockerfile | Builds a reproducible Playwright+Node+Yarn image aligned with the repo and CI expectations. |
| tools/e2e/Dockerfile.dockerignore | Introduces a strict allowlist build context to keep image builds small and predictable. |
| tools/e2e/docker-compose.yml | Defines the base container run (platform pin, worker cap, output mounts). |
| tools/e2e/docker-compose.update.yml | Adds an overlay for snapshot updates to mount sources back into the working tree. |
| tools/e2e/assert-browsers.js | Fails image build if Playwright browser revisions don’t match the installed Playwright version. |
| playwright.config.ts | Adds validated worker override logic for Docker/CI and documents screenshot stability constraints. |
| .github/workflows/e2e.yml | Runs E2E in Docker (no runner-side browser install) and uploads the report artifact. |
| .github/workflows/e2e-approve-snapshots.yml | Regenerates snapshots inside the same Docker image and commits PNG updates. |
| docs/guides/06-testing.md | Documents Docker-based visual testing workflow and worker behavior. |
| packages/e2e/README.md | Documents Docker requirement/usage for screenshots and snapshot updates. |
| AGENTS.md | Updates contributor guidance to use Docker for visual E2E runs and snapshot updates. |
| packages/components/icon/e2e.ts | Adjusts the icon e2e page to render a dropdown-item inline with the required styles. |
| packages/components/icon/e2e.playwright-spec.ts | Adds a deterministic wait for SVG icon hydration before screenshots. |
| packages/components/code-block/e2e.playwright-spec.ts | Waits for async highlighting and marks the heavy test as slow() for stability. |
| .eslintignore | Ignores Playwright output directories that are produced by local Docker runs. |
| .prettierignore | Ignores Playwright output directories to avoid formatting generated artifacts. |
| .gitattributes | Enforces LF for Docker build inputs and declares PNGs as binary to avoid EOL/filter damage. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
|
Visit the preview URL for this PR (updated for commit dbbc81b): https://koobiq-next--prs-1889-ybv5gaa1.web.app (expires Sun, 16 Aug 2026 10:35:36 GMT) 🔥 via Firebase Hosting GitHub Action 🌎 Sign: c9e37e518febda70d0317d07e8ceb35ac43c534c |
docker compose creates a missing bind-mount source itself, but the daemon does it - so against a rootful daemon playwright-report/ and test-results/ land in the working tree owned by root, and the developer's next run cannot write into them. It does not heal itself either: the ownership survives until someone with sudo removes the directories, and the error it eventually produces comes from inside the container, pointing nowhere near the cause. Both CI workflows already carried a `mkdir -p` for exactly this, but tools/e2e/run.js did not, so a plain local run stayed exposed. Creating them in the wrapper covers every supported entry point, and the workflow steps go away rather than duplicating it.
| // receives the panel's styles through the overlay. Pulling them in here is what keeps it looking | ||
| // like a dropdown item — which is the point of the capture, since the icon inside it is what is | ||
| // being checked. | ||
| styleUrls: ['../dropdown/dropdown-tokens.scss', '../dropdown/dropdown.scss'], |
There was a problem hiding this comment.
не совсем честный тест получается, кажется что стоит его просто отрефакторить
There was a problem hiding this comment.
Согласен, убрал блок целиком — dbbc81b.
Честно этот item отрендерить тут нельзя. kbq-dropdown-item живёт только внутри оверлея, поэтому прошлая версия открывала дропдаун и ловила ту часть панели, которая случайно попадала в bounding box компонента: на старом baseline она обрезана правым краем. Плюс съёмка шла наперегонки с анимацией открытия — именно это и делало тест флаки. Вариант с inline-item менял одну подделку на другую: стили панели импортировались в страницу иконок через styleUrls, чтобы кнопка просто выглядела как item.
Внешний вид item с иконкой уже покрыт e2e дропдауна, в настоящей панели. Теряем только иконку, отрезолвленную в inline SVG, а не в font-class — разница слишком тонкая, чтобы держать ради неё любую из двух подделок.
Заодно ушёл KbqDropdownModule из imports страницы, baseline перегенерирован в контейнере.
It could not be rendered truthfully. A kbq-dropdown-item only exists inside an overlay, so the previous version opened the dropdown and captured whatever of the panel happened to overlap the component's bounding box - clipped at the edge, and timed against the open animation, which is what made the test flaky in the first place. Rendering the item inline instead traded that for importing the dropdown's own stylesheets into this page to fake the appearance, which is no more honest. The appearance of an item with an icon is already covered by the dropdown suite, in a real panel. What is lost here is the icon resolved as inline SVG rather than as a font class, which is a thin enough distinction not to be worth either fake.
No description provided.